Skip to content

fix(kafka): avoid recording incorrect partition when using random partitioner#4627

Open
gauravSsinha wants to merge 1 commit into
open-telemetry:mainfrom
substrai:fix/kafka-instrumentation-incorrect-partition
Open

fix(kafka): avoid recording incorrect partition when using random partitioner#4627
gauravSsinha wants to merge 1 commit into
open-telemetry:mainfrom
substrai:fix/kafka-instrumentation-incorrect-partition

Conversation

@gauravSsinha
Copy link
Copy Markdown

Description

Fixes incorrect messaging.kafka.partition attribute in producer spans when using the DefaultPartitioner without an explicit partition or key.

Problem

When no explicit partition and no key are provided, the DefaultPartitioner randomly selects a partition. The instrumentation calls instance._partition() to record this value in the span, but the actual send() method calls _partition() again independently, potentially selecting a different random partition. This causes the span's messaging.kafka.partition attribute to not match the actual partition where the message was delivered.

Fix

Only compute partition in the instrumentation when the result is deterministic:

  • Explicit partition provided → use it directly
  • Key provided → partition is determined by key hash (deterministic), safe to compute
  • Neither provided → omit the messaging.kafka.partition attribute rather than record an incorrect value

This is more correct than recording a potentially wrong partition, which could mislead debugging and monitoring.

Changes

  • KafkaPropertiesExtractor.extract_send_partition() — refactored to only call _partition() when the result is deterministic
  • _enrich_span() — handles None partition gracefully by omitting the attribute

Fixes #4625

…titioner

When no explicit partition and no key are provided, the DefaultPartitioner
randomly selects a partition. The instrumentation was calling _partition()
to record this value in the span, but the actual send() method calls
_partition() again independently, potentially selecting a different
partition. This caused the span's messaging.kafka.partition attribute to
not match the actual partition where the message was delivered.

Fix: only compute partition in the instrumentation when the result is
deterministic — either explicitly provided by the caller or determined
by key hash. When partition would be randomly assigned, omit the
attribute rather than record an incorrect value.

Fixes open-telemetry#4625

Signed-off-by: Gaurav Kumar Sinha <gaurav@substrai.dev>
@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented May 26, 2026

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: gauravSsinha / name: gauravSsinha (408adce)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Incorrect partition in opentelemetry-instrumentation-kafka-python

1 participant